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ABSTRACT 

The report summarizes findings and recommendations of 
a California Task Force on Program Effectiveness in Special 
Education. The 30 Task Force members ("stakeholders" in special 
education) were divided into committees to address four goals: (1) 
identify the mission ct special education and describe the context in 
which special education programs operate; (2) delineate major desired 
outcomes for handicapped scudents; (3) specify ef roctiveness 
indicators for special education programs; and (4) recommend 
procedures for determining and ensuring .special education program 
effectiveness. A conceptual model was adopted with 
planning-evaluation cycles, evaluation of student outcomes, and 
relationships between effectiveness indicators and student outcomes. 
The :uission of special education is defined, and the provision of 
programs that increasingly lead to improve student outcomes is 
identified as central. The section on the context of spenial 
education notes demographic trends, economic climate, ana educational 
policy issues. Twenty-tv/o effectiveness indicators art icientified, 
clustered into the categories of curriculum, instructional setting, 
and instructional processes. Desired student outcomes are organized 
into developmental/academic factors, personal/social achievements, 
and community /economic factors. Finally, the major goals and 20 
recommendations are offered. Appendixes provide information on 
effectiveness indicators and norm-referenced testing. Eleven 
references are provided. (DB) 
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Introduction 

The Task Force on Program Effectiveness in Special Education, appointed by 
Patrick Campbell, Assistant Superintendent of the California State Department of 
Education, was charged with developing a conceptual framework for planning and 
evaluating the quality of special education programs and services across the state. Task 
Force members represented a broad cross section of special education "stakeholders" 
within California. Included were SELPA directors, special education teachers, state 
consultants, program coordinators, district directors of special education, special 
education administrators of county offices, program specialists, speech and language 
specialists, professors, parents, directors of private schools, assistant and associate 
superintendents of local education agencies, program evaluators, nurses, occupational 
therapists, school psychologists, physical therapists and resource specialists. Several of 
the members served as designated representatives of various associations and commis- 
sions; others were selected to represent the general field of special education. 

These members were divided into four committees structured around the major 
goals identified by the Task Force. These goals were as follow: 

1. Identify the mission of special education and describe the context in which 
special education programs operate. 

Special education programs are developed in response to a particular mis- 
sion and operate in a complex of contextual variables that affect the nature 
and success ">f service delivery. The mission and context of special education 
provide a background against which to plan and evaluate programs for 
students with handicapping conditions. 

2. Delineate major desired outcomes for handicapped students. 

The central focus of program effectiveness must be on the expected results or 
outcomes for the students. Specification of broad student outcomes repre- 
sents the core of the conceptual model developed by the Task Force. 



ERIC 



II 



pagel 



Model for Program Quality in Special Education 



3. Specify effectiveness indicators for special education programs. 

Research and policy formulations within general and special education, as 
well as existing listings of quality indicators, are sources for identifying 
program characteristics associated with effective special education. 

4. Recommend procedures for determining and ensuring special education pro- 
gram effectiveness in California. 

Using the conceptual model for program effectiveness as a basis, the Task 
Force recommendations to the State Department of Education set the direc- 
tion for future planning and evaluation of special education ax\d related 
services. 

The work of tliese committees, over a 15 montii period of time, has resulted in the 
development of a model for quality special education. This model provides a concept 
tual basis for evaluating spedal education programs and services. 

This advisory report from the Task Force describes the conceptual model in detail. 
The first chapter presents a schematic of the model and briefly outiines the key features 
of this schematic. The next four chapters delineate these features: the mission of special 
education, the context in which special education operates, effectiveness indicators, and 
targeted student outcomes. The final chapter includes recoTiimer dations from the Task 
Force on Program Effectiveness to the Division of Spetic! Ed uca don, California Depart' 
ment of Education, for adopting and implemenilxig the Model for Program Quality in 
Special Education 
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Conceptual ModeH ( ..lity Special Education 

The Task Force was charged with developing a model to use in determining to 
what extent special education i; achieving its mission. One type of evaluation model 
the Task Force might have suggested would involve identifying a series of "tests" to be 
administered to various parts of the special education system, specifying how the test 
results should be interpreted, and perhaps, prescribing how and to whom the results 
vere to be reported. This univariate approach to special education evaluation has the 
potential of establishing a sort of general healtti screening of the system and might, 
depending on the results, be useful in generating future support of special education 
from the legislature and the general commimity. "One-shot" models run the risk, 
however, of merely showing that the system is or is not achieving its stated mission and 
goals. They do not provide direction for needed program improvements and they are 
particularly poor choices for establishing the effectiveness of irmovative practices. 

Because improvement in special education is likely always to be a goal, the Task 
Force adopted a model in which planning and evaluation are intimately tied to program 
and in which the focus is on student outcomes. A schematic of the conceptual model 
developed by the Task Force is presented in Figure One. Emphasized in the schematic 
aro the key elements that characterize quality special education programs. These 
elements include: (a) planning-evaluation cycles; (b) an emphasis of evaluation on 
student outcomes; and (c) a relationship between effectiveness indicators and student 
outcomes. The program m^^^al exists within a a context of internal and external 
variables that affect how special education programs are planned, managed and 
evaluated. These variables include such considerations as social/political factors, 
demographic trends, the economic climate and educational policy issues, among 
others. The planning and evaluation cycles respond to an enduring program mission 
which underlies the development of special education programs and services for 
students with handicaps. 
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Planning-Evaluation Cycles 

Systematic planning-evaluation is a aitical feature of this program quality model. 
In the view cf the Task Force, quality special education programs are those which 
continually improve on the basis of built-in planning and evaluation processes. In its 
emphasis on continued plrjming and evaluation, the Task Force's model differs rather 
dramatically from many program evaluation models, including the ''one-shof model 
previously described. The approach presented here conceptualizes evaluation not as 
a requirement imposed on the special education system, but as a continuous and 
systematic aspect of the system. Implicit here is the idea that effectiveness in special 
education is not something (ti^t is accomplished; rather, it is a continuous state of 
achieving. 

^klthough Figure One depicts a single planning and evaluation cycle, the Task 
Force conceptualized the planning and evaluation as involving multiple, overlapping 
cycles. This differs dramatically from perspectives of evaluation as an occasional, a 
periodic, or i terminal event. For instance, current Program Quality Reviews and 
Coordinated Compliance Reviews, conducted by the California State Department of 
Education, are either annual or triennial. Although such long term cycles seem to be 
appropriate for certain questions, they clearly are not appropriate for others. This is 
particularly true if planning-evaluation is viewed as a continuous aspect of quality 
special education. At the individual student level, for example, planning and evalu- 
ation should include, but not be limited to, the annual individualized education 
program (IE?) cycle. Certain goals will require weekly evaluation cycles and some may 
require even shorter cycles. Similarly, at the school-site level, goals such as those having 
to do with the school climate are not likely to be appropriately implemented if the 
planning-evaluation cycle is annual. It may be that monthly cycles would better 
address these concerns. In other words, planning-evaluation should be a central feature 
of special education, with the duration of planning-evalualion cycles being appropriate 
to the questions to which planning-evaluation is addressed. 
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Student Outcomes 

The emphasis of evaluation/ as conceptualized in the Model for Quality Special 
Education, is on student outcomes. The model is not specifically concerned with the 
evaluation of compliance with legislative guidelines/ nor Is it primarily concerned with 
the evaluation of educational policies and practices. Instead, the emphasis is on the 
product of special education. Are students acquiring the knowledge and skills they need 
to succeed bothinandoutof school? Focusing as it does on student outcomes, themodel 
is closely articulated with the mission of special education as well as the current national 
emphasis on accountability in the schools. 

The model's emphasis on student outcomes encompasses measures throughout 
the student's formal education well as at program completion. The dual focus is on 
what students are doing in school and what they are prepared to do after leaving school. 
Included are achievement in all curricular areas as well as such considerations as the 
extent to which sHidents like schooling and the extent to which they attend school. At 
the post-secondary level, attention is directed to what students actually do after jeaving 
school. In other words, measures of both short-range and long-range outcomes are the 
indices of program effectiveness that constitute the focus of planning and evaluation 
cycles. 

Short-range outcomes include those that teachers, parents and others could deter- 
mine on a daily or weekly basis. Tliey revolve around such questions as the following: 

• Is the student achieving objectives consistent with his/her Individualized 
Education Program (lEP) goals? 

• Does the student have appropriate access to general education programs and/ 
or interactions with non-handicapped peers? 

• Is the student's program appropriately aligned to the district's course of study? 

• the student acquiring increasingly complex concepts and behaviors? 
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Student success in achieving short-range outcomes should be predictive of attain • 
ing the more long-range, or terminal, goals and objectives. Relevant summative 
evaluation questions as students approach and enter adulthood might include: 

• Has the student attained a reasonable level of independence? 

• Is the student prepared to participate in integrated adult environments? 

• Has the student met minimum proficiency standards? 

• Has the student attained a high school diploma or other form of recogiution for 
program completion? 

• Does the student display appropriate levels of achievement in the community? 
Does he/she work, pay taxes, live independently, contribute to the community, 
demonstrate good citizenship, maintain self-esteem? 

• Does the student seek additional educational opportimities? 

Effectiveness Indicators 

The Task Force's model also emphasizes the relationships between certain special 
education practices, called effectiveness indicators, and sti^dent outcomes. Such effective- 
ness indicators are more or less continuously being discovered, lefined, and reported 
in pro^essiuna. publications. Practices are regarded as effectiveness indicators when 
they consistently contribute to improved student outcomes. Loosely speaking, effec- 
tiveness indicators may be thought of as independent variables that influence student 
outcomes, the dependent variables. Certain evaluation measures, including the Pro- 
gram Quality Review of tl\e State Department of Education, the Los Angeles County 
Characteristics of Effective Special Education Programs (1986), and the MidSouth Regional 
Resource Center's Effectiveness Indicators in Special Education (1986) concentrate on 
whether schools and school districts have these indicators in place. 

The Task Force model focuses on student outcomes, with the presumption that 
effectiveness indicators contribute to these outcomes. When program evaluation data 
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reveal tJiat programs are not attaining targeted student outcomes, spedal educators 
should st?arch out desaiptiov ' of effectiveness indicators, including the listing that 
appears later in this document. Ftom these listings practices can be identified which, 
if implemented properly, would grestly improve student outcomes. Further, effective- 
ness indicators should be viewed as practices which are neither perfectly correlated 
with student outcomes nor implemented in precisely the same way time after time. 
Consequently, to the extent possible, all such practices should be tailored to the spedfic 
setting or situation and continued only as long as justified by student outcome 
evaluations. Finally, special educators should systematically describe and evaluate 
ti\elr own practices, retaining only those that yield desirable student outcomes. 

Program effectiveness indicators can be studied through formative evaluation, 
involving self-study and/or independent review teams, which describe program 
characteristics and determine the extent to which key effectiveness indicators are in 
place. This level of evaluation can address sn-*^ issues as what are the features of the 
program, what relationships can be documented between the effectiveness indicators 
and expected learner outcomes, and to what extent student outcomes influence pro- 
gram practices. 
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The Mission of Special Education 

The mission of education in American society has traditionally been to inculcate 
the sodal and cultural mores and values of the society and to provide the opportunity 
for social and economic mobility of its dtizens. This mission, and the educational 
objectives that articulate it, is based on the democratic ideal that all individuals have a 
right to an education that will enable them to achieve their maximum potential. Public 
Law 94 -142, the Education for All Handicapped Children Act, embodies this philoso- 
phy. It states: 

It is thepurpose of this Act to assure that handicapped children have available to them 
. . Mfree,ttppropriatepubliceducationwhichernphasizesspecialeducationandrelated 
services designed to meet their unique needs, to assure that the rights of handicapped 
children and their parents or guardians are protected, to assist states and localities to 
provide for the education of all handicapped children and to assess and assure the 
effectiveness of efforts to educate handicapped children. 

The wording of California's Education Code 56001(a) reflects a similar commit- 
ment: 

Each individual with exceptional needs is assured an education appropriate to his or 
her deeds in publicly supported programs through completion of his or her prescribed 
course of study or until such time that he or she has met proficiency standards 

This commitment is supported by the California Special Education Division's ar- 
ticulation that its mission is to provide leadership and assistance to ensure that a "free, 
appropriate, and quality public education prepares all students with exceptional needs 
for employment, citizenship, independent living, and personal, academic and social 
growth." 

The Task Force identified as a central mission of special education the provision 
of programs that increasingly lead to improved student outcomes. These programs 
may be provided through a continuum of service delivery option. . 'anging from highly 
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segregated to fully integrated settings The placement of individual students along this 
continuum will vary with the nature and the severity of the handicapping condition. 
For some, special education will be provided m the form of support services to the 
gene^ral class prog.'am, with the curricular objectives and content identical to that of 
general education. For others, special education may requive the reformulation of 
educational objectives and content and may involve delivery of programs away from 
the regular classroom environment. 

In 1987, Thomas Bellamy, Director of Special Education Programs of the U.S. 
Office of Special Education and Rehabilitative Services, emphasized that special edu- 
cation reflects a particular value system with implicit purposes and goals. He 
recommended that the effectiveness of special education be assessed relative to the 
extent to which handicapped students leave school with the same options as everyone 
else: 1) vocational success; 2) a reasonable place to live; and 3) a friendship network. 
Bellamy concluded that data in these three areas, plus reductions in special education 
dropouts, would indicate that special education is a good place in which to invest 
money for the handicapped. For handicapped students, therefore, the long-range 
purpos/e and mission of special education remains the same as ;hat for all of education, 
namely to maximize the potential of individuals to participate in and make contribu- 
tions to our social system, and to do so as independently as possible. 
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The Context in Which Special Education Operates 

Special education, as well as general education, exists within a context of complex 
social/political, demographic, economic and educational policy variables. The extent 
to which either special or general education is able to fulfill its mission and attain its 
goals is influenced by these attitudes, values, policies and trends. Factors such as 
demographic changes, economic recession or depression, changing roles for education, 
fiscal support for education and technologic^ advances are but some of the issues that 
affect the extent to which special education can be effective in achieving its mission. It 
is within this broad context that the mission of special education is formulated, program 
goals implemented, and program plaiming and evaluation conducted. 

Demographic Trends 

Numerical, ethnolinguistic and social-economic changes in the population bring 
pressures to bear on all aspects of public education. Sizeable increases in the numbers 
of school-age children who are from diverse linguistic and cultural groups are occur- 
ring. Already more than 50 percent of the students in the K-3 classrooms in California 
are from non-majority group backgrounds (California State Department of Education, 
1986). In addition to a growing Hispanic population, there are large numbers of 
children from Pan-Padfic and Middle Eastern backgroimds. It can be expected that the 
proportion of children from these groups that qualify for special education services will 
be approximately the same (about ten percent) as for the general population. However, 
in neither general education nor special education are there the numbers of bilingual/ 
bicultural professional personnel or the proven curricula md strategies to ensure full 
educational opportimity for these diverse groups. These variables attenuate student 
outcome attainment for both handicapped and non-handicapped minority group 
students. 



ERIC 



page 11 



Model for Program Quality in Special Education 



Other changes in the population that are bringing pressures to bear on public 
education include the increased numbers of 1) teenage pregnancies and drug/alcohol 
dependent motliers/ 2) infants bom without benefit of prenatal care, and 3) the 
inaeasing proportion of children who live in poverty (Education Week, 1988). These are 
all factors associated with inaeased risk for birth defects and handicapping conditions. 
Further/ improvements in medical technology and care are resulting in increased 
numbers of premature/low weight infants and multiply handicapped infants who 
survive. All these factors in combination mean that there is an inaeasing need for 
educational support services/ although/ in recent yearS/ budget constraints have re- 
sulted in reductions in these support services in many districts. 

Expanded conceptualizations of least restrictive environment and the movement 
toward deinstitutionalization and community-based living have meant that schools are 
providing programs for students with a broader range of handicapping conditions and 
severity of disabilities than in the past. At the same time/ the extension of special 
education services at both ends of the age continuum/ e.g./ transition services into early 
adulthood and intervention programs to at-risk infants and children/ represent an 
additional increase in the special education population. These considerations all have 
an impact on the planning and evaluation process. 

Economic Climate 

The general economic picture of the state and nation strongly influences program 
effectiveness. It affects the fiscal support for education programs in general and for 
compensatory and similar education support programs in particular. Factors such as 
class size, the availability of special programs and equipment and the recruitment of 
credentialed teachers are affected. In addition/ economic indicators affect the avail- 
ability of jobS/ the disaetionary income that individuals enjoy/ the numbers of those 
who Uve in poverty, and other quality of life factors. 
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Clearly, the criteria for successful or effective educational programs vary with the 
fiscal health of society in general. For example, economic productivity is typically a 
targeted outcome for speda^ education students. However, in times of widespread 
unemployment within the general population, it may be that program effectiveness 
should not be restricted to data on actual employment, but should include measures 
that encompass the acquisition of empkyment skills as well. Further, for some students 
with severe handicaps, the desirability of paid employment may be diminished if the 
income from employment makes them ineligible for Supplemen tal Security Income and 
Medi-Cal support. Here an important outcome might be the ability to ascertain when 
to work and how to live a productive work life outside the context of significant levels 
of paid employment. Evaluation of employment data must consider the effect on 
student outcomes of economic factors such as these. 

Medical and scientific advances are additional contextual variables that affect 
special education services. They inaease both the survival rate of infants and young 
children with severe handicaps and their potential to achieve a qualitative life. New 
technologies are providing important tools for disabled individuals. They can place the 
curriculum within the reach of students who cannot read, write, see or hear and they 
also can enhance the employability and independence of individuals with handicaps. 
Miaocomputer technologies lower the costs and increase the effectiveness of some 
forms of instruction, including compensatory and remedial instruction. In short, the 
plethora of technological advances hold the promise of diminishing the handicapping 
nature of many disabilities and normalizing the life and occupational options available 
to individuals with handicaps. 

Educational Policy Issues 

Recent attention to the reform and improvement of education has led to an 
increased emphasis on the rigor of American education. For example, California Senate 
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Bill 813, the Hughes-Hart Educational Reform Act of 1983, has increased the number of 
instructional minutes per day and instructional days per year, required a greater 
number of academic courses for graduation, and calls for higher grading standards for 
some courses. As a part of the educational reform effort in California, SB 813 directs the 
Superintendent of Public Instruction to delineate a course of study with a wide range 
of specific competencies. At both the elementary and secondary levels, model curricu- 
lum standards and frameworks in each of the major curriculum areas have been 
developed, with the California Assessment Program (CAP) designed to measure 
progress toward achievement of these standards. 

The goal of this curricular specification is the attainment of sldlls in the following 
areas: (a) basic skills, including language, reading, writing, spelling, mathematics, 
science and social sciences; (b) communication skills; (c) social/interpersonal skills, 
including positive attitudes toward self and others; (d) health and physical education; 
(e) prevocational and vocational skills; (0 art, music and other forms of creative 
expression; (g) citizenship; and (h) the development of higher order thinkmg skills. At 
the district level, the course of studies specifies a sequence of goals and objectives 
consistent with the State Curriculum Guidelines. A high level of cooperation between 
special education and general education is required to insure that both the curricula and 
instructional strategies afforded students with handicapping conditions are reflective 
of the imibrella core curriculum standards and goals to the fullest extent possible, while 
at the same time providing flexibility for individual differences. 

Cutbacks in other support and compensatory programs within the schools, 
coupled with the higher achievement standard, have increased the pressure on special 
education to provide support to a larger portion of the school population. One response 
to these pressures has been the movement toward increased integration of general and 
special education programs. Consolidation of categorical programs, encouraged by 
Assembly Bill 777, the School-Based Program Coordination Act (1981), permits the co- 
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ordinating, integrating and merging of categorical funds, resources, etc through alter- 
native funding models. The intent is to provide efficient and cost-effective education 
to special needs students, to eliminate redundancy and to reduce intrusions on the 
general school programs. 

Within special education, academic and social integration is reflected in the 
philosophy of Least Restrictive Environment (LRE). The LRE construct centers on 
assisting every student to become as independent as possible, capable of interacting 
productively with non-handicapped persons and living a normal life within society. 
California's LRE policy (1986) sets forth that individuals with exceptional needs typi- 
cally should attend the same school as non-handicapped students in their neighbor- 
hoods, except when the lEP team determines such a setting to be inappropriate to the 
studenf/s educational and social needs. This policy further states, 'Tlacement other 
than in a regular education dassroom shc^ild be considered only when the lEP Team 
determines that the regular environment, services and/or curriculum cannot be modi- 
fied effectively to meet the needs of the student as specified in his/her lEP." The policy 
also requires that a full continuimi of program options is available to identified 
students. 

Within the past few years, transition programs have received increased attention. 
The goals of these programs are to assist handicapped students in making a successful 
transition to the world of work and in assuming independent living responsibilities. 
Local districts address the vocational needs of handicapped students in a variety of 
ways, including adopting and/or adapting portions of their regular vocational educa- 
tion curricula, coordinating with regional occupational programs and offering sup- 
ported employment programs. Relevant preparation of handicapped students for 
employment is complicated by the changing nature of occupations. Twenty-five years 
ago, the half-life of occupations was seven years, meaning that in this period of time, a 
worker could expect 50 percent of job tasks to change. Today, the half-life of a job is 14 
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months (Patterson, 1987). This means that past training quickly becomes obsolete and 
that frequent opportunities for retraining of handicapped individuals are required. 

The extent to which any program or service contributes to achieving its mission is 
related to the availability of qualified personnel 1^ deliver that program. Special 
education is experiencing significant shortages of personnel, due in part to program- 
matic growth of two percent a year overall (Califonua State Department of Education, 
1986) with a stable nine percent of the total exu'ollment served by special education 
(Advisory Commission on Special Education, 1986). Teacher attrition also contributes 
to persoxmel shortages. This attrition is influenced by teacher, student and program- 
matic variables such as teacher retirements, salary competition from other fields, the 
credentialing structure, the inaeasing severity of the handicapping conditions served 
and the increasing cultural-lingiustic diversity within the student population. Wide- 
spread shortages in the numbers of fully qualified teachers and classroom aides 
diminishes the potential for program effectiveness. 
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Effectiveness Indicators 

During the several years immediately following the passage of PL 94-142, rapid 
changes in the educational system resulted in improved access to public education for 
students with handicaps. With the accomplishment of this major objective, the focus of 
attention at both state and national levels has moved beyond compliance with basic 
mandates to a quest for excellence in programs and services provided handicapped 
students. A variety of publications and research studies have attempted to identify the 
full range of factors associated with quality spedal education programs. These efforts, 
which have included examinations of the major bodies of research and opinion in both 
general and spedal education, have led to the identification of many such factors. For 
instance. Effectiveness Indicators in Special Education, a document resulting from a 
massive effort spearheaded ^ y the MidSouth Regional Resource Center (1986), identi- 
fied over 600 indicators of program effectiveness; the Los Angeles County Office of 
Education's Characteristics of Effective Special Education Programs (1986) also identified 
several hundred indicators of program quality. The indicators identified in both 
document3 include structural, administrative and instructional considerations that 
affect the quality of programs and services for students with handicaps. 

For purposes of this report, the Task Force narrowed its focus to specific program 
features which appear to be most directly associated with the attainment of the targeted 
student outcomes. It limited its consideration to issues of curriculum/ the structure of 
the instructional setting, and the processes by which instruction is accomplished. This 
compilation should be considered neither fully comprehensive nor permanent. Ongo- 
ing research will continue to add to empirical knowledge, leading to future modifica- 
tions of this listing. 

The 22 effectiveness indicators included in this report are clustered Into three 
categories: 1) Currioilum; 2) Instructional Setting; and 3) Instructional Processes. 
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1. Curriculum 

Each Local Education Agency (LEA) has a comprehensive course of study that is 
continuously undergoing revision in light of the skills and knowledge students 
require, research findings concerning scope and sequence of skills and knowledge, 
and the textbook adoption cyde. 

1.1 Thecourseof study consists of a comprehensive and sequenced setof goals and 
objectives that students are to achieve as a result of instruction in development 
tal/acadenuc skills, personal/sodal skills, and community/economic skills. 

1.2 The course of study indudes goals and objectives as well as provisions for 
adapting materials and modifying instruction to address the needs of students 
with all types and levels of disability. 



2. Instructional Settings 

LEA administration, induding site administration, organii^ and fadlitates school 
activities so as to encourage effective instruction for all studenls. 

2.1 There is evidence of collaboration between general and special education 
teachers and other staff in the planning for and provision of educational 
opportimities to all spedal needs students. 

2.2 A full continuum of education and related service options is available and 
utilized to accommodate all spedal needs students; selection of options is 
based on student outcome data. 

2.3 Attractive and inviting learning environments are fadlitated through both 
LEA polides and student and staff activities. 

2.4 Each LEA provides opportimities and encouragement for consistent interac- 
tion and communication among teachers and parents. 

2.5 The pupil-teacher ratio varies with the intensity and range of handicapping 
concitions exhibited by the shidents in a particular setting. 

2.6 There is a process in place whereby documentation of student outcomes and 
changes in student placement are used to accomplish the goal of teaching 
students in the least restrictive environment. 

2.7 Spedal education services are delivered as dosely as feasible to 2 mainstream 
school and conununity environments appropriate to the educational charac- 
teristicu and needs of the individual student. 
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2.8 Special education classes and general education classes are located within 
school sites in a manner which facilitates integration, mainstreaming and 
social interaction. 

2.9 All students have equal access to school facilities, equipment and materials, as 
appropriate and extra-curricular activities or programs. 

2.10 Parent education programs/ services are available to families of general and 
special education students. 

2.11 General and special education teachers and other program staff demonstrate 
the necessary skills and training for instructing the shidents they are assigned. 

2.12 Consultation approaches characterize the interactions among special educa- 
tion, general education, other categorical programs and relevant community 
agencies. 

2.13 Ongoing inservice training is designed and revised on the basis of its effective- 
ness in producing student outcomes and is made available for general and 
special education teachers and other program staff. 

2.14 The instruction provided by general and special education teachers and other 
program staff is observed regularly and feedback concerning these activities is 
provided in a way that leads to greater instructional effectiveness. 

3. Instructional Processes 

Instructional strategies are based on evidence of tiieir effectiveness in produdng 
desired student outcomes. 

3. 1 Placement in the course of stuay Iz based on curriculum-referenced assessment 
and predictions concerning the student's rate of growtii in particular skill 
areas. 

3.2 Assessment and instruction are consistent with the lEP goals and objectives 
and aligned with the course of study. 

3.3 Teaching occurs as a series of short "assess-teach-assess" cycles. 

3.4 Teaching maximizes appropriate skill generalization. 

3.5 A management system is used which reflects each student's progress through 
the course of study. 
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3.6 The instructional materials utilized to optimize student learning are selected 
on tiie basis of data which demonstrate their effectiveness. 

Each of these effectiveness indicators can be viewed as independent variable 
related targeted student outcomes. Should a program fall short of expectations, these 
are the practices to be examined in an effort to improve program effectiveness. In their 
current form, the indicators lack the specificity necessary for prescriptive program 
development efforts. Appendix A provides illustrations of the more precise interpre- 
tations that must be developed to guide program improvement efforts. One Task Force 
reconunendation, included in the final chapter of this report, calls for the State Depart- 
ment to fund the development of a product, or products, that comprehensively 
delineates each of the effectiveness indicators such that they can serve as a guide in 
plaxming and evaluating special education programs and services. 
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Student Outcomes 

The primary purpose of special education is to produce individuals who are 
independent, contributing and participating members of society. This means that 
students must acquire skills in die developmental/ academic, personal/sodal and 
economic/community arenas. 

Independent individuals are tltose who demonstrate economic self-sufficiency in 
the community and are able to accommodate their basic life needs. Independent indi- 
viduals exhibit work-related behaviors and life and career planning abilities which 
result in their being engaged in training and /or gainful employment. Special education 
is effective when it assists handicapped students to realize their individual and unique 
potentials for productive work, emplo3rment and commimity living in the mainstream 
of society. 

Contributing individuals are those who exhibit a variety of personal/sodal skills 
that enhance thequality of their lives and the lives of others. These personal/sodal skills 
manifest themselves in positive peer relations, quality family partidpation, good healtii 
and safety habits, self-care and responsibility for their living environments, sodal 
interactions and communications witii others, dtizenship, reaeation and leisure pur- 
suits and positive self-image. 

Participating individuals are those who take an active part in their homes, 
communities and work environments. In addition to the attributes discussed in the 
preceding paragraphs, the broad array of academic and cultural literacy skills encom- 
passed within the core curriculum contributes to full sodetal partidpation. 

Spedfic skills assodated with the attainment of these outcomes fall under the 
categories of developmental/ academic factors, personal /social achievements, and 
community/economic skills. The first of these, developmental/academic factors, 
reflects the acquisition of the developmental milestones, the core curriculum, cognitive 
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skills and functional living skills. Personal/ sodal achievements are those evident when 
individuals are able to relate to and communicate with others, to gain peer acceptance 
and to attain self fulfillment. Community/economic factors empower individuals to 
work, live and enjoy recreation in the co^-^munity and to become fully participating 
citizens. Included are work behaviors^ specific job skills; generic skills for lieeking, 
securing and maintaining employment; skills for obtaining and maintaining independ- 
ent living situations/ utilizing community resources and participating in community 
activities. 

Measures of Student Outcomes 

Student progress toward these outcomes should be assessed from early childhood 
through adulthood in order to determine the effectiveness of program practices. These 
data can guide ongoing program planning and evaluation cycles, including daily 
instructional practices, lEP development and overall program design and revision. 
Categories of outcome m«iasures are listed for each of the three realms of student 
achievement. Included in brackets are possible approaches for gathering information 
in each of the areas listed. It should be noted that across all areas, the use of norm- 
referenced tests with handicapped students is of limited value due to such shortcom- 
ings as lack of a direct relationship to core curriculum, the inflexibility of their 
administration for handicapped students and the lack of universal use. Appendix B 
provides a discussion of the util ity of such i^ieasures with special education students. 

Developmental/Academic Factors 

1. Developmental Progress 

Indices include documentation of the progress of handicapped individuals in the 
following domains: cognition; communication; social interaction; fine and gross 
motor; self-help; and family interactions. [Parent interviews; direct testing meas- 
ures using various commercially available instruments and behavioral observa- 
tions]. 
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2. Academic Achievement 

The extent to which special education r tudents attain functional skills, basic 
i^cademic proficiencies, thinking skills and cultural literacy is an indication of the 
extent to which the program is effective in achieving the goals of education. 

2.1 Criterion-referenced and curriculum-related measures, as well as other 
measures of individual student progress provide specific information on 
student skiUs acquisition. [District curriculum expectancies; course 
completion. Note: The Comprehensive Assessment Systems currently being 
developed and piloted may become an informal source in this area.] 

2.2 Norm-referenced achievement tests are typical measures of these factors for 
the general school population. [Note: Appendix B provides a discussion of 
the drawbacks of norm-referenced achievement test scores as measures of 
program effectiveness in special education.] 

3. Grade Level Fromotion 

Student progress, as well as the rate of progress, through the grade levels provides 
an index of academic achievement. [Review of school records.] 

4. Minimum Frofidendes 

Student acquisition of district minimum proficiencies, the rate of this acquisition, 
and the method used to demonstrate ttiese profidcndes are relevant indices. 
[Review of school records; district profidency standards.] 

5. Graduation/Program Completion 

The percentages of students who either graduate from secondary school or 
complete their designated curricula is a rdated criterion. [School record data on 
percentage of students for each program who meet graduation requirements 
through the regular and/or differential standards as well as for each the percent- 
ages of students attaining some other recognition of program completion.] 

6. Post-Secondary Training Participation 

The percentages of students who enroll in post-secondary training and the levels 
of success attained by them is an index of program effectiveness. [Parent and 
student reports; follow-up studies.] 
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Personal/Social Achievements 

1. Peisonal/Social Skills 

Student attainment of personal /social skills in a variety of behavioral areas are 
encompassed under this heading: health, leisure, interpersonal skills, self-esteem 
and other aspects of affectivedevelopment. [Observational ratmgs and interviews 
with the students and their parents and/or employers; measures of self-esteem, 
family and community participation, and degree of independence in living.] 

2. Participation 

The extent to which handicapped individuals successfully participate in class, 
school, community and extracurricular activities are indicative of skill attainment 
in this outcome category. [Review of student records; teacher, parent, student 
reports; follow-up studies.] 

3. Attendance Patterns 

The school attendance patterns of handicapped students in both their regular and 
special education programs indicate levels of personal/sodal skill attainment. 

3.1 Daily attendance of students with handicaps, including partial day atten> 
dance, suggest a level of development in the person/sodal skills area. [Daily 
attendance records.] 

3.2 Remaining in school (not dropping- out) is another index of success in 
this area. [School site records; district pupil count records.] 

Community/Economic Factors 

1* Independent Living Ski Is 

Student success in acquiring independent living skills which allow them to access 
community services, live independently, recreate, and express their rights as 
citizens constitute an important outcome category. [School records; behavioral 
observations and check-lists; follow-up studies; reports/interviews of parents, 
students and teachers.] 

2. Work Competencies 

The demonstration of work competencies and occupational skills required for 
employment and/or work productivity is a dimension of economic productivity. 

2.1. One set of criteria would relate to the development of work-related compe- 
tencies (e.g., work behaviors, study skills, on-task behavior, sper • revo- 
cational skills) while in the school program. [Review of lEPs; . 'istic 
observation; parent, student, teacher interviews; check sheets; r' lata- 
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bases; anecdotal records. Note: The Job Training Partnership Act (JTPA) and 
similar related efforts may result in useful locally developed aiteria. Work 
competencies also could contribute to these efforts.] 

2.2. Another set of criteria could include the acquisition of occupational compe- 
tencies. [Ascertainmg completion of specific training programs and compe- 
tency verification of acquired occupational skills.] 

3. Employment Record 

The number or percentage of students who obtain employment provides an index 
of the attainment of major program goals and outcomes. Measures of work 
productivity must be sensitive to part-time and non-salaried /volunteer endea- 
vors/ as well as to the overall economic climate. 

3. 1 Factors such as current and past employment status, salary, promotions, job 
retention, reasons for leaving jobs and others are hdpful in ascertaining the 
level of attainment in this area. [Follow-up studies; student and parent 
report; employer interviews.] 

3.2 These measures should be longitudinal and not applied only immediately 
after high school graduation. [Longitudinal studies with common design 
funded in representative regions across the state; analysis of data in terms of 
regional characteristics and economic climate considerations.] 

As indicated by the foregoing listing, the model's focus on student outcomes does 
not restrict consideration only to summative evaluation at the point of program 
completion. Student progress is to be continually measured so that program adjust- 
ments can be made, when indicated, to ensure progress toward the ultimate attainment 
of the target outcomes. Figure Two illustrates the skills included in each competency 
area and the longitudinal nature of evaluation required. Some developmental/ 
academic factors should be assessed from early childhood into adulthood. Others (such 
as demonstration of minimum competencies and enrollment in and graduation from 
secondary and post-secondary programs) are of relevance only with older students. 
Personal/social skills, which develop over a lifetime, as do independent living and 
work competencies, must be assessed from early childhood through age 21 and beyond. 
Employment, the last factor depicted, can be assessed only beginning in late adoles- 
cence and early adulthood. 
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FIGURE TWO 



MEASURES OF STUDENT OUTCOMES 



PrCSChCtti Klemtntary secondary Pnst Segimdary 



Developmen till/ Academic 

Developmental measures - > 

Achievement testing > 

Grade level promotion > 

Minimum proficiencies > 

Graduation/program > 

completion 
Post secondary training 

participation > 

Personal/Social 

Personal/social skills - > 

Participation in class, school, - > 

community and extra- 
curricular activities 

Attendance in both regular - - - > 

and special programs 

Com muDlty /Economic 

Independent living skills > 

Work competencies - > 

Work behaviors, including 
volunicerism and other 

productive efforts > 

Employment > 



T 



K jU: The dotted lines represent a sequence of skills and outcomes for each area. Progress along each sequence 
should be measured at least periodically and, In some instances, continuously. 
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Task Force Recommendations 

The model described in the preceding chapters provides a conceptual framework 
in which to consider questions of the quality and effectiveness of special education 
programs. Within the context of a program planning and evaluatior^ paradigm, tlie 
model focuses on student outcomes and delineates effectiveness indicators associated 
with the attainment of expected student outcomes. The key issue in program 
effectiveness is whether or not special education is achieving its program mission and 
goals. Systematic planiung and evaluation is a means of monitoring program outcomes 
and continuously striving toward program improvement. 

The Task Force on Program Effectiveness recommends that the California Depart- 
ment of Education, Special Education Division, take a leadership role in rethinking the 
wa3rs in which program effectiveness is conceptualized and evaluated. It should do 
this, in part, by taking the following actions: 

1. Adopt the Model for Quality Special Education as the basis for a longitudinal/ 
visionary approadi to program improvement and long range planning. 

1.1 Use the Model as a framework for continuous planiung and evaluation. 

1.2 Advocate planning and evaluation cycles as an integral component of special 
education at school site, LEA, and state department levels. 

1.3 Develop guidelines to assist LEAs in specifying program goals that communi- 
cate expf!Ctations for student progress and learning outcomes in a maimer that 
guides the designof curriculum and theselection of instructional materials and 
strategies. 

1.4 Promulgate techniques of planning and evaluation by developing integrated 
evaluation approaches that encourage the State Department of Education to 
work more closely with LEAs and individual school sites to improve program 
quality and effectiveness. 
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1.5 Develop procedures for involving in the planning and evaluation processes all 
stakeholder groups, including those involved in the lEP development and 
monitoring, school-site planning and self-study, independent program quality 
reviews, program evaluation, and statewide evaluations of special education 
services. 

1.6 Provide necessary funding to implement program evaluation consistent with 
the conceptual model. 

1.7 Revise Coordinated Compliance Reviews and Program Quality Reviews to 
include the plaiming and evaluation elements reflected in the The Program 
Effectiveness irt Special Education Task Force Report: Model for Program Quality in 
Special Education. 



2. Disseminate the conceptual model to all stakeholders. 

2.1 Provide copies of the The Program Effectiveness in Special Education Task Force 
Report: Model for Program Quality in Special Education to LEA directors of special 
education, school superintendents, community agencies, parent and advocacy 
groups and other stakeholders. 

2.2 Develop guidelines for utilizing the model at all levels of program planning 
and evaluation. 

2.3 Provide information and training about the model at appropriate conferences, 
institutes and workshops. 

2.4 Develop a system for providing technical assistance to LEAs for implementing 
the model to address statewide questions as well as local conditions, resources 
and informational needs. 



3. Develop products and procedures to facilitate utilization of the model. 

3.1 As a separate project fund thedevelopmentof aproductorproducts to describe 
and delineate effectiveness indicators and to serve as a guide to program 
planning and evaluation. 

3.2 Develop valid and reliable measures appropriate for evaluating specific stu- 
dent outcomes as outlined in this report. 

3.3 Develop guidelines for integrating planning and evaluation efforts across all 
age levels (infants and toddlers, preschoolers, school-aged children and ado- 
lescents, and young adults) and across handicapping conditions. 
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4. Develop infonnation management systems to collect necessary data on student 
outcomes both in school and after program completion. 

4.1 Develop in the State Department of Education a student-level database that is 
interactive between LEAs and the State Department. 

4.2 Coordinate program evaluation efforts at LEA and State Department levels in 
a manner consistent with this model. 

4.3 Wherever possible, align the special education reporting systems with regular 
education reporting procedures and requirements [e.g., pupil count forms, 
California Basic Educational Data Systems (CBEDS), etc.]. 



5. Implement the Model 

5.1 Select specific student outcomes in developmental/academic, personal/sodal 
and community /economic areas as the focus of program evaluation. 

5.2 Assist LEAs to review and refine program practices based on student outcome 
data. 

5.3 Develop reporting procedures through which evaluation findings are dissemi- 
nated in a timely and efficient manner to all stakeholders and interested 
parties. 
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Delineation of Effectiveness Indicators 

Recognizing that so-called "effectiveness indicators" are of limited utility in the 
forms in which they typically are presented, the second reconmiendation of the Task 
Fc/ce was that the State Department of Education fund the development of a product 
or products to describe and delineate the effectiveness indicators such that they can 
serve as a guide to program planning and evaluation. Either printed or mediated 
documents could provide txie level of specification required for the effectiveness 
indicator statements to become uiseful guides to program planning and evaluation. 

This appendix includes samples of the level of specification that is required. These 
examples are not comprehensive, nor are they intended to be. Rather, they are included 
to provide a sense of tlie task that must be imdertaken and completed if the effectiveness 
indicators developed by the Task Force are to play a meemingful role in the design, 
delivery and evaluation of special education programs and services in the State of 
California. 

Sample Effectiveness Indicators 
Instructional Settings 

2.9 Students have equal access to school facilities, equipment, materials, and extra- 
curricular activities or programs, as appropriate. 

EXAMPLES 

• School Facilities: Every effort is made to educate students at neighborhood 
schools. Students haveequal access to all buildings and rooms within the school, 
including playing fields (i.e., interactive recess and P.E.), cafeterias, auditoriums 
and offices. 

• Equipment: Students have equal t ccess to school equipment to include, but not 
limited to, typewriters, computers, office equipment, dispensing machines and 
audio-visual equipment. 
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• Materials: Students have their own copies of textbooks used in the general 
education courses in which they receive instruction. Art, science, shop, and 
home economics materials are available for special education students. Special 
education teachers, as well, have their own copies of teacher's guides, resource 
materials, etc., which are used by general education teachers in teaching this 
subject matter. 

• Extra-Cuiricular Activities or Programs: Students are informed about and en- 
couraged to participate in extra-curricular activities or programs. LEAs docu- 
ment the degree of student involvement and determine what factors affect this 
involvement (i.e., prerequisites for student government, clubs, dances, band, 
drill team, chorus, athletic events, etc.). 



Instructional Processes 

3.5 A management system is used which reflects a student's progress through the 
course of study. 

The purposes of the management system is: 1) to ensure a direct correlation 
between the district course of study and the instruction sMdents are receiving in 
special education; and 2) to provide a monitoring system to ensure that students 
are afforded equal access to the core curriculum. 

The management system should be generic enough to apply to all students 
regardless of their handicapping condition. 



EXAMPLES 

(See the following two pages) 
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Management Profile Form 



Grade 3 
Books 



Student Name 



Date of Birth 



ID Number 



Skill Xrea 

Addition and Subtraction to 10 



Addition and Subtraction to 18 



Problem Solving— Using Your 
^Skills 



Geometry 1 



Obi ect Ives 

Master the addition facts 
through 10 



Master the subtraction facts 
related to sums through 10 



Master the addition facts 
through 18 



Master the subtraction facts 
related to sums through IB 



Add three addends having sums 
at 20 or less 



Solve word problems involving 
addition facts for s uns 
through 18 



Solve word problems involving 
?u')traction facts related to 
sums through 18 



Identify a point and a line 
segment 



Recognize squares, rectangles « 
triangles, and circles 



Hasteey and Date 



1 



4& 
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Studant MaiM 



Date o£ Birth 



I. D. Number 



Spelling 
Brown Level, Qrad* 3 

Management Profile Form 



SUll 



unit 1 Leeeon 1 



Leeeon 4 
Leeeon 5 
Lesson 6 



^esson 



Unit 2 Lusson 10 



unit 3 



Lesson 11 



t.«sson 
LiUfin. 



Lesson 



Lesson IS 



Lga»on 
Lesson 



Words ipsllsg with sllut e 



2L 



Plurals uaino m or 



f~ lurals of consonant y words 
rreomar oiuraia 



Lesson 9 SDellino and the process of wrlteing" 



words that end with double 
eqnsonants 
rdfl wit 



Lth double consonants in the 



Wor 

r r; "i^Mi- 

Lesson 12 jgaat tense formed with eA 
LasMon 13 vmut sense ot final y wori 
Lesson 1 4 Irreor-' -l l ■ 



woros 



Review 

spelling ana th« process of writin<a 



DM. 



Lesson 19 words a««l}^ "^«^!; 
Lesson 20 Words apeiled witK 



or 




The word 



[ ending r 



er 



er_ 



^fhe wora yyi±na« « 
words analled with mt 
words aoelled with ioh 



Lesson zS Revj.ew 
Lesson 27 SpellinTTui the process o 



School 
Year Mastered 



* iUMSiLM Words anelle d with ailant t«tf ra 

Lesson 2 9 The pra^ix un 



Lesson i6 



Lesson -- 
.tesfpff 32 
Lesson 33 



The prefix un 
The prefix 



contractions 

xne poaa essxve form ot nounT 



words speiiea with double 



Leeson 34 
Leaaon XT 



conaonantM » im 
c ompound words 



Leaaon 35 

Lesson 36 spelling and the proceaa of writing 
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Achievement Test Data and the Evaluation 
of Special Education Programs 

Public school special education programs serve a diverse group of learners. This 
diversity is evident in the range of handicapping conditions included under the rubric 
of special education and in the continuum of severity that characterises each type of 
exceptionality. It is also reflected in the continuum of services that educaUonal agencies 
make available to individuals with exceptional needs: services that span the spectrum 
from residential placement in special schools or hospitals to consultant assistance to 
general education teachers to facilitate classroom curricular adaptations. 

Evaluation of the effectiveness of such a service delivery system cannot be 
approached from a single perspective. Just as federal law requires that no one 
procedure be used as the sole criterion for the determination of the educational program 
for handicapped children and youth, no one index of effectiveness is sufficient to serve 
as a measure of the worth of such a complex human services system. 

If efficiency were the only concern, the choice of measures for the evaluation of 
special education would be a simple decision. Existing measures would be selected, 
preferably ones already in use by school systems for the evaluation of some portion of 
the general education program. Standardized achievement tests, administered regu- 
larly thiough the grades, would be a logical choice. In California, that choice would 
likely reduce to the Basic Skills Test of the California Assessment Program (CAP) which 
is administered annually to representatives of grades 3, 6, 8, 10, and 12. 

In the evaluation of educational programs, however, efficiency is not the only 
concern. Efficient means of data collection that produce spurious results are neither 
useful nor economical. Although standardized tests of academic achievement may 
serve to support the contention that students identified as handicapped and served in 
special education do not perform at a level commensurate with that of grade peers, that 
is all they will do. The results are flawed for the purposes of program evaluation. 
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There are several reasons for this. First, the domains jampled by standardized 
tests of academic achievement are school skill domains. While these may be appropri- 
ate for students with the milder degrees of some handicapping conditions, they are not 
appropriate for all students served in special education. Many such students do not 
pursue academic curricula; their educational goals instead stress such areas as oral 
language development, self-help and daily living skills, vocational preparation and 
behavior and social skills. Assessing tiieir progress in academic realms would only 
show tiiat they had not learned that which they had not been taught, hardly a valid 
measure of program effectiveness. 

Second, standardized tests of academic achievement are group-administered 
measures. This type of adnunistration unduly penalizes many handicapped students. 
On group tests, students often must work imder timed conditions, write answers on a 
separate answer sheet, work independentiy, monitor their own behavior, sustain 
attention to test tasks and read test questions, even when reading is not the skill area 
under investigation. Because of these demands, group measures tend to produce a low 
estimate of tiie performance of mildly handicapped students (McLoughlin & Lewis, 
1986). 

Third, academic achievement tests are typically normed on a general school 
population, with little or no information on the number or typeof handicapped students 
included in the standardization sample (Fuchs, Fuchs, Benowitz, & Barringer, 1987). 
Also, because handicapped students are characterized by school performance prob- 
lems, the scores they earn fall within the lowest ranges of test norms. Very low (or very 
high) test scores tend to be less reliable than scores within the average range (McLough- 
lin & Lewis, 1986). To paraphrase Cohn, Cohn, and Kanevsky (1988), because a small 
number of items actually discrimuiate individual differences at tiie lower levels of 
ability, random influencet or error can account for moro variation among the scores 
earned by members of tiie less able group tiian among the more typical group. In 
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addition, there is the threat of statistical regression of extreme scores toward the mean 
(Campbell & Stanley, 1963). Given these potentialities for error, it is obvious that group- 
administered tests of achievement normed on typical school populations are not an 
appropriate choice for gathering psychometrically sound data on changes in the skill 
levels of handicapped students. 

Even less appropriate than nationally-normed tests of achievement is the CAP test 
used in California. This test utilizes a matrix sampling procedure so that students take 
only a portion of the test. Results are then aggregated to obtain school and district 
averages. To employ this measure as an index of the effectiveness of special education 
programs would require substantial alterations in test content to reflect special educa- 
tion curricula, modification of administration procedures (with the development of a 
range of adaptations tailored to the diverse needs of the handicapped population), and 
the development of new sets of norms reflecting the types and severity levels of diverse 
kinds of learning problems. Such revisions would likely result in a less adequate 
measure of achievement in the general education population and a measure that 
remains inappropriate to assess the effectiveness of special education programming. 

In summary, group-administered standardized tests of academic achievement are 
best viewed as scret'ning measures, measures whose results prompt general educators 
to attend to the potential school performance problems of students falling within the 
lower score ranges. They are not accurate measures of ac'^ual skill development in 
students with handicaps, nor are they appropriate tools for evaluating the effectiveness 
of special education programs. 
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